1. Field of the Invention
The present invention relates to image photographing apparatuses, image photographing methods, and computer programs for capturing images containing subjects, such as human beings, and more particularly, to an image photographing apparatus, an image photographing method, and a computer program for performing, in response to recognition of a particular pattern of a subject contained in an image, automatic processing relating to camerawork.
More specifically, the present invention relates to image photographing apparatuses, image photographing methods, and computer programs for automatically determining a photographing timing by utilizing recognition of a smile of a subject contained in an image, and more particularly, to an image photographing apparatus, an image photographing method, and a computer program for, in the case that an input image contains a plurality of subjects, focusing on an appropriate subject, performing smile detection on the subject, and automatically determining a photographing timing.
2. Description of the Related Art
Cameras have a long history as devices for recording visual information. Recently, instead of silver-salt cameras for performing photography using films and sensitive plates, digital cameras for digitizing images captured using solid-state image capturing elements, such as charge-coupled devices (CCDs) or complementary metal-oxide semiconductors (CMOSs), have been widely used. Digital cameras have advantages of being capable of storing digitized and encoded images in a memory, performing image processing and image control using a computer, and not having a problem relating to the lifetime of films.
Almost the same basic camerawork is adopted for silver salt cameras and digital cameras. Together with the digitization of captured images, automation technology for camerawork has been developed. Thus, even users who are inexperienced at performing camera operations are significantly less likely to perform unsuccessful photography. The automation technology for camerawork includes, for example, an auto-focus (AF) function, an auto-exposure (AE) function, and motion blur correction. Digital cameras have an advantage of being capable of implementing processing, such as AF and AE, not only by using an optical system but also by digital processing. In addition, in a case where image-quality adjustment is performed on a captured image by digital processing, such as application of white balance gain to a captured image using auto white balance (AWB), colors can be reproduced accurately.
With functions such as auto-focus and image-quality correction, processing is to be performed with reference to any one of one or more subjects existing in an image frame. Formerly, users set important subjects by manual operations. Recently, however, image capturing technology utilizing face detection for detecting a face from a captured image and automatically focusing on the detected face or automatically correcting image quality with reference to the detected face has been introduced.
As a technique for detecting a face from a captured image signal, for example, a technique for performing matching between an input image signal and an average face image used as a template has been widely available in the technical field to which the present invention pertains (see, for example, Japanese Unexamined Patent Application Publication No. 2003-271933 (paragraphs 0051 to 0053 and FIG. 6)). In a case where a face in a captured image is tilted, it is difficult to achieve accurate face detection. Thus, a technique for detecting the angle of tilt of a face, rotating the face image in accordance with the detected value, and then performing matching processing has been available. As a method for detecting the angle of tilt of a face, for example, a technique for detecting candidate center lines of the face, calculating a degree indicating the plausibility of each of the candidate center lines being the actual center line of the face on the basis of the bilateral symmetry of the face, and outputting the angle of the center line having the highest plausibility as the angle of tilt of the face has been available (see, for example, Japanese Unexamined Patent Application Publication No. 9-171560 (paragraphs 0032 to 0044 and FIG. 4)).
In addition, recently, research and development relating to automatic photography for determining a photographing timing by utilizing recognition of a particular pattern, such as a smile, from an image of a subject has been conducted.
For example, a camera that determines whether a target portion of a main subject to be photographed meets a predetermined photographing condition and that outputs a photographing timing signal at a time when the target portion of the main subject meets the photographing condition has been suggested (see, for example, Japanese Unexamined Patent Application Publication Nos. 2000-347278, 2004-294498, and 2006-237803). The term “target portion” represents, for example, in the case of photographing a human being, the shapes and colors of the eyes and the mouth of the human being, and in the case of photographing a bird, the feathers of the bird.
As specific criteria for determining a photographing timing, conditions under which an image of a human being can be captured finely, such as “not blinking”, “looking in the direction of a camera”, “smiling”, and the like may be regarded as being criteria for determining an appropriate photographing timing. In addition, technology for improving a photograph of a human being as a subject has been available. However, it is difficult to modify a photograph of a person who is blinking such that the person is not blinking or to modify a photograph of a person who is not laughing such that the person is laughing. This is one of the reasons why “smiling” is set as a photographing condition. For example, the photographing condition “smiling” can be determined on the basis of image information, such as the areas of white portions of the eyes of a subject, the lateral length of the mouth of the subject, and the area of a white portion in the mouth.
In the case of performing photography, it is necessary to adjust the focus and the iris to be suitable for a subject. As described above, with the automatic photography function for recognizing a particular pattern, such as a smile, of a subject and performing a photographing operation, after a smile is detected, an operation for causing the focus and the iris to follow the subject is performed and the photographing operation is then performed. That is, after the smile is detected, a period of time is necessary, the period of time being the same as that in the case of taking a single picture using a normal auto-focus function.
At least a time from about 0.2 seconds to about 0.3 seconds is necessary for the operation for causing the focus and the iris to follow the subject, whereas, in general, a human smile may last only about 0.1 seconds. Thus, even if a smile can be detected from a subject image, an actually captured image often does not contain the smile.